Skip to content

Support configuring nomad/consul/ebi versions#2868

Open
djeebus wants to merge 9 commits into
mainfrom
support-more-versions
Open

Support configuring nomad/consul/ebi versions#2868
djeebus wants to merge 9 commits into
mainfrom
support-more-versions

Conversation

@djeebus
Copy link
Copy Markdown
Contributor

@djeebus djeebus commented May 30, 2026

Support for customizing Nomad/Consul and source image versions.

@cla-bot cla-bot Bot added the cla-signed label May 30, 2026
@cursor
Copy link
Copy Markdown

cursor Bot commented May 30, 2026

PR Summary

Low Risk
Changes are limited to image build tooling and install scripts; defaults preserve current behavior unless overrides are set.

Overview
Nomad cluster disk image builds can now be driven from the environment: Consul and Nomad versions pass through optional PACKER_* make variables into existing Packer defaults, and the base OS image is configurable on AWS (AMI name filter) and GCP (source image) instead of being fixed only in HCL. Install scripts use unzip -o so re-running unpack during image builds does not fail on existing files in /tmp.

Reviewed by Cursor Bugbot for commit 892346d. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request parameterizes the source AMI filter name and Packer build variables, and updates the Consul and Nomad installation scripts to use the unzip -n flag. However, using -n prevents overwriting existing binaries in /tmp, which could silently result in outdated or incorrect versions being installed; it is recommended to use the -o flag instead to force overwriting.

Comment thread iac/nomad-cluster-disk-image/setup/install-consul.sh Outdated
Comment thread iac/nomad-cluster-disk-image/setup/install-nomad.sh Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented May 30, 2026

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
2706 3 2703 7
View the full list of 3 ❄️ flaky test(s)
github.com/e2b-dev/infra/tests/integration/internal/tests/api/sandboxes::TestSandboxListPaginationRunningLargerLimit

Flake rate in main: 42.38% (Passed 775 times, Failed 570 times)

Stack Traces | 93.7s run time
=== RUN   TestSandboxListPaginationRunningLargerLimit
    sandbox_list_test.go:327: Created sandbox 1/12: irui06twkzd67vsvbs48k
    sandbox_list_test.go:327: Created sandbox 2/12: ifkecqj90yj4e3e3nlsf4
    sandbox_list_test.go:327: Created sandbox 3/12: iim3b2x7o27s6rpe6qbby
    sandbox_list_test.go:327: Created sandbox 4/12: i93250kxzavrrbnxhejv2
    sandbox_list_test.go:327: Created sandbox 5/12: id8nctbj68asvq6jgcnfw
    sandbox_list_test.go:327: Created sandbox 6/12: im8mf19m2aldz97h0jwen
    sandbox_list_test.go:327: Created sandbox 7/12: id9mmujxfzgfpihof2xw4
    sandbox_list_test.go:327: Created sandbox 8/12: ijzxlugz1xd3sg07jheh3
    sandbox_list_test.go:327: Created sandbox 9/12: ipod5c04e4825rk0t8lb6
    sandbox_list_test.go:327: Created sandbox 10/12: imulc3y5cgw64jp05prxa
    sandbox_list_test.go:327: Created sandbox 11/12: i0yxs11zeun06jz4jfgkr
    sandbox_list_test.go:327: Created sandbox 12/12: ipnnw1jw4rr3v31w9xytr
    sandbox_list_test.go:330: 
        	Error Trace:	.../api/sandboxes/sandbox_list_test.go:340
        	            				.../hostedtoolcache/go/1.26.3.../src/runtime/asm_amd64.s:1771
        	Error:      	"[]" should have 12 item(s), but has 0
    sandbox_list_test.go:330: 
        	Error Trace:	.../api/sandboxes/sandbox_list_test.go:330
        	Error:      	Condition never satisfied
        	Test:       	TestSandboxListPaginationRunningLargerLimit
--- FAIL: TestSandboxListPaginationRunningLargerLimit (93.69s)
github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity

Flake rate in main: 57.21% (Passed 768 times, Failed 1027 times)

Stack Traces | 65.6s run time
=== RUN   TestSandboxMemoryIntegrity
=== PAUSE TestSandboxMemoryIntegrity
=== CONT  TestSandboxMemoryIntegrity
    sandbox_memory_integrity_test.go:27: Build completed successfully
--- FAIL: TestSandboxMemoryIntegrity (65.62s)
github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity/tmpfs_hash

Flake rate in main: 57.32% (Passed 758 times, Failed 1018 times)

Stack Traces | 195s run time
=== RUN   TestSandboxMemoryIntegrity/tmpfs_hash
=== PAUSE TestSandboxMemoryIntegrity/tmpfs_hash
=== CONT  TestSandboxMemoryIntegrity/tmpfs_hash
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{start:{pid:1259}}
Executing command bash in sandbox icapsk8gl978q1heyb74n (user: root)
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Total memory: 985 MB\nUsed memory before tmpfs mount: 190 MB\nFree memory before tmpfs mount: 794 MB\nMemory to use in integrity test (60% of free, min 64MB): 476 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"476+0 records in\n476+0 records out\n499122176 bytes (499 MB, 476 MiB) copied, 2.3337 s, 214 MB/s\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"\tCommand being timed: \"dd if=/dev/urandom of=/mnt/testfile bs=1M count=476\"\n\tUser time (seconds): 0.00\n\tSystem time (seconds): 2.31\n\tPercent of CPU this job got: 99%\n\tElapsed (wall clock) time (h:mm:ss or m:ss): 0:02.33\n\tAverage shared text size (kbytes): 0\n\tAverage unshared data size (kbytes): 0\n\tAverage stack size (kbytes): 0\n\tAverage total size (kbytes): 0\n\tMaximum resident set size (kbytes): 2636\n\tAverage resident set size (kbytes): 0\n\tMajor (requiring I/O) page faults: 2\n\tMinor (reclaiming a frame) page faults: 345\n\tVoluntary context switches: 3\n\tInvoluntary context switches: 12\n\tSwaps: 0\n\tFile system inputs: 176\n\tFile system outputs: 0\n\tSocket messages sent: 0\n\tSocket messages received: 0\n\tSignals delivered: 0\n\tPage size (bytes): 4096\n\tExit status: 0\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Used memory after tmpfs mount and file fill: 672 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_memory_integrity_test.go:70: Command [bash] completed successfully in sandbox i0hr22jzihq51ovjvxabk
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{start:{pid:1275}}
Executing command bash in sandbox irxftkazkoo9ihwzcygnv (user: root)
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{data:{stdout:"acc9302146fcc702a9008bf6206b723492da79caec8265f3696a0cff15a1e340\n"}}
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_memory_integrity_test.go:80: Command [bash] completed successfully in sandbox i0hr22jzihq51ovjvxabk
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{start:{pid:1278}}
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
Executing command bash in sandbox i0hr22jzihq51ovjvxabk (user: root)
    sandbox_memory_integrity_test.go:110: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:81
        	            				.../hostedtoolcache/go/1.26.3.../src/runtime/asm_amd64.s:1771
        	Error:      	Received unexpected error:
        	            	failed to execute command bash in sandbox i0hr22jzihq51ovjvxabk: unavailable: HTTP status 502 Bad Gateway
    sandbox_memory_integrity_test.go:110: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:78
        	            				.../tests/orchestrator/sandbox_memory_integrity_test.go:110
        	Error:      	Condition never satisfied
        	Test:       	TestSandboxMemoryIntegrity/tmpfs_hash
--- FAIL: TestSandboxMemoryIntegrity/tmpfs_hash (195.48s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Comment thread iac/provider-aws/nomad-cluster-disk-image/Makefile Outdated
Comment thread iac/nomad-cluster-disk-image/setup/install-consul.sh Outdated
Comment thread iac/provider-aws/nomad-cluster-disk-image/Makefile Outdated
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d227a32. Configure here.

Comment thread iac/provider-gcp/nomad-cluster-disk-image/Makefile Outdated
@sitole sitole force-pushed the support-more-versions branch from 7345499 to 892346d Compare June 1, 2026 09:43
@sitole sitole changed the title support configuring nomad/consul/vault/ebi versions Support configuring nomad/consul/ebi versions Jun 1, 2026
@sitole sitole marked this pull request as ready for review June 1, 2026 09:44
@sitole sitole self-requested a review June 1, 2026 09:48
Copy link
Copy Markdown
Member

@sitole sitole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I just unified env vars naming (892346d). Feel free to revert if there was any reason not to.

Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — build-tooling plumbing only; defaults preserve current behavior and prior bot feedback (empty-var override, AWS/GCP var name mix-up, source_name typo) is fixed in the latest commits.

Extended reasoning...

Overview

This PR makes the Nomad cluster disk image build configurable via env vars. It touches: install scripts (install-consul.sh, install-nomad.sh) to add unzip -o for idempotent extraction; AWS and GCP Packer Makefiles to conditionally forward PACKER_CONSUL_VERSION, PACKER_NOMAD_VERSION, network/subnet identifiers, and a base-image override; and the corresponding variables.pkr.hcl / main.pkr.hcl files to introduce source_ami_filter_name (AWS) and source_image (GCP) variables wired to the existing hardcoded defaults.

Security risks

None meaningful. All changes are scoped to image-build tooling that runs at build time (not at runtime in the sandbox or control plane). No auth, crypto, network policy, or permission code is touched. The new variables only influence which OS base image is used and which Consul/Nomad versions get installed.

Level of scrutiny

Low. This is mechanical configuration plumbing for image builds. The $(if $(VAR),...) Makefile pattern ensures empty overrides are not passed to Packer, so existing defaults in the HCL files continue to apply when the new env vars are unset — i.e., the change is backward compatible by construction.

Other factors

Earlier commits drew several bot findings from Cursor and Gemini (empty env vars overriding Packer defaults, AWS Makefile inheriting GCP variable names, an incorrect source_name vs source_image mismatch, and a unzip -n instead of -o). The latest commits in the branch ("Unify packer envs vars naming", "fix env var", "oops, this is the right flag", "go back to the old vars") address each of these — the current diff uses vpc_id/subnet_id for AWS, source_image for GCP, conditional var passing, and unzip -o. No bugs were surfaced by the bug-hunting system for the current commit, and there are no unresolved human review threads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants